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MOBILE WIRELESS COMMUNICATION HANDSET 
WITH SOUND MIXER AND METHODS THEREFOR 

FIELD OF THE DISCLOSURE 

[0001] The present disclosure relates generally to mobile wireless 
communication devices, and more particularly to handheld cellular 
communication devices capable of mixing and playing polyphonic audio 
files and methods therefor, 

BACKGROUND 

[0002] Cellular handsets having sound mixers are known generally. 
In the P503i i-mode cellular telephone available from NTT DoCoMo, for 
example, the user must first select several soundtracks and then manually 
mix the selections. Particularly, in the P503i cellular telephone, the user first 
selects a music Category (Rock, Dance, Pops, etc.) soundtrack and then first 
and second Backing Pattern soundtracks in corresponding menus displayed 
sequentially after each selection. In the P503i, the selected soundtracks are 
mixed manually only after making all selections. Thus in the P503i cellular 
telephone, the user cannot listen to more than one selected soundtrack at a 
time during the soundtrack selection process until after making all 
selections and manually mixing, since only individual soundtracks are 
played during the selection process. The P503i cellular telephone also 
permits the user to adjust the key and tempo of the mix, but the selected 
changes in these characteristics cannot be heard until after making the 
selections and manually mixing. 
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[0003] The various aspects, features and advantages of the present 
disclosure will become more fully apparent to those having ordinary skill in 
the art upon careful consideration of the following Detailed Description 
with the accompanying drawings described below. 

BRIEF DESCRIPTION OF THE DRAWINGS 

[0004] FIG. 1 is an exemplary mobile wireless communication handset. 

[00051 FIG. 2 is an exemplary process flow diagram for implementing 
various aspects of the inventions. 

[0006] FIG. 3 is an exemplary soundtrack data set file stored on the 
handset. 

[0007] FIG. 4 is an exemplary audio mix data reference file. 

DETAILED DESCRIPTION 

[0008] FIG. 1 is a handheld mobile wireless communication device 
comprising generally a processor 10 and a DSP 12, memory 20, for example 
ROM, RAM, EEPROM, etc., a transmitter and receiver 30, inputs 40, for 
example, a keypad and/ or joystick, microphone etc,, outputs 50 including 
an audio output and an output signal connector interface, and a display 60. 
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[0009] The mobile wireless communication handset may be for 
example a cellular telephone, personal digital assistant, one-way or two-way 
pager, or some other handheld mobile communication device. 

[00010] The disclosure is drawn generally to methods for dynamically 
creating polyphonic audio mixes on handheld mobile wireless 
communication devices, and generally to methods for creating and storing 
polyphonic mixes using limited data storage resources on wireless 
communications handsets, and generally to methods for integrating audio 
data reference files and soundtrack data files in the form of a polyphonic 
audio format file for uploading from wireless communications handsets, 
and combinations thereof. 

[00011] The disclosure is implemented generally in software on 
handheld mobile wireless communication devices, including polyphonic 
audio synthesizer software. In one embodiment, the audio synthesizer is a 
MIDI standard synthesizer, known by those of ordinary skill in the art In 
other embodiments, the synthesizer may be of another standard or a 
proprietary audio file synthesizer. The implementation of audio 
synthesizers on cellular handsets is known generally as discussed above in 
the Background of the instant specification. 

[00012] The software is invoked by a user input command. In one 
embodiment, a polyphonic audio mix is created on a handheld mobile 
wireless communication device by selecting a first soundtrack, and playing 
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the first soundtrack upon selection thereof. In the present disclosure , 
unless indicated otherwise,, the playing of a soundtrack or sound effect of 
one or all of the soundtracks of the polyphonic mix occurs upon selection 
thereof, without additional input by the user. 

[00013] While one soundtrack is playing, for example, a first selected 
soundtrack, a subsequent or second soundtrack is selected and played along 
with the one or more prior selected soundtracks. Thus by selecting and 
playing each subsequently selected soundtrack while one or more 
previously selected soundtracks are playing, the polyphonic audio mix is 
mixed dynamically as it is complied, which provides the user with an 
immediate indication of how the composition of the selected soundtracks 
will sound during creation. 

[00014] In one implementation, soundtracks are mixed by selecting and 
de-selecting combinations of soundtracks, in any order. The selected 
soundtracks are mixed dynamically upon selection and unmixed upon de- 
selection until the user compiles a satisfying combination of soundtracks, 
which may include different sound effects discussed more fully below. The 
compiled audio mix may be saved or discarded upon completion of the 
selection exercise. 

[00015] In FIG. 2, a first soundtrack is played upon its selection at 210. 
Then, while the first soundtrack is playing, a second soundtrack is played 
along with the first sound track upon selection of the second soundtrack at 
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220, etc. The selected soundtracks may also be de-selected while playing 
upon re-selecting the same sound track at 210. For example, the first 
soundtrack is played upon selection, wherein re-selection of the first 
soundtrack will result in de-selection of the first soundtrack- The process of 
selecting and de-selecting soundtracks continues in no particular order until 
an appealing combination of sounds aremixed, whereupon the user may 
save the mix at block 230, as discussed more fully below. 

[00016] According to the process illustrated in FIG* 2, several 
soundtracks may be mixed. For example, a first soundtrack is selected and 
played, and then a second soundtracks is selected and played with the first 
soundtrack while the first soundtrack is playing. Additional soundtracks 
may be selected and played while the first and second soundtracks are 
playing, etc. Any one or more of the selected soundtracks, for example, the 
first soundtrack, may subsequently be subtracted from the mix, while the 
mix is playing, for example by de-selecting the first soundtrack. 
Alternatively, all of the selected soundtracks may be deselected 
simultaneously. 

[00017] In one embodiment, a first soundtrack is selected from a first 
plurality of soundtracks, and a second soundtrack is selected from a second 
plurality of soundtracks. Additional soundtracks may be combined, or 
mixed, with the first and second tracks, as desired. 
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[00018] 



In one embodiment, at least one of the soundtracks is a base, or 



reference, soundtrack selected from a plurality of different base 
soundtracks. In a preferred embodiment, the reference soundtrack is 
selected before selecting any other soundtracks. Thereafter, one or more 
soundtracks are selected subsequently and mixed dynamically while the 
reference soundtrack and any other selected soundtracks are playing. Other 
embodiments are devoid of a base soundtrack. 

[00019] The plurality of different reference soundtracksmay be 
characterized generally by corresponding rhythmic and harmonic 
characteristics- Exemplary characteristics are those associated with Jazz, 
Rhythm & Blues, Soul, Rock, Latin, and Classical and Big Band, among 
other types of music. 

[00020] Some of the soundtracks may be selected from a group of 
soundtracks having corresponding melodies. In one implementation, a 
melodic soundtrack is selected from a plurality of musical instrument 
soundtracks, for example, from either a percussion, or a wind, or a string 
instrument soundtrack. In another embodiment, each melodic soundtrack 
has corresponding variations. For example, upon selection of a "piano" 
soundtrack, there may be several piano soundtrack variation sub-selections 
among which to choose. 

[00021] In embodiments where there is a reference soundtrack, the 
instrument or melodic soundtracks available for selection may be 
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dependent upon, or specific to, the reference soundtrack selected. Where 
Rhythm & Blues is selected as the reference track, for example, the 
instrument soundtracks available may be piano, drum, bass and horn and 
corresponding variations thereof. Where the reference soundtrack is Rock, 
different instrument soundtracks and corresponding variations are 
available. 

[00022] In another embodiment, an audio characteristic or effect is 
selected for a selected soundtrack while playing the soundtrack. Exemplary 
sound effects include echo and reverberation, among others. De-selection of 
a soundtrack for which one or more effects have been selected, also de- 
selects the corresponding effects. 

[00023] The audio characteristic of the soundtrack is changed while the 
soundtrack is playing upon selecting the audio characteristic, thus 
providing the user with an immediate indication of how the selected 
characteristic affects the soundtrack as the polyphonic file is created. After 
selecting a soundtrack, a sound characteristic, or effect, for the selected 
soundtrack is selected and played. Generally more than one global effect 
may be changed or selected. Selection and de-selection of effects are also 
illustrated in FIG. 2. 

[00024] In some embodiments, a global audio characteristic common to 
all of the selected soundtracks is selected and played while playing the 
selected soundtracks, thus changing the audio characteristic of all of the 
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selected soundtracks while the soundtracks are playing. Generally more 
than one global effect may be changed or selected. 

[00025] In some embodiments, the number of soundtracks selected is 
limited/ for example, to 4 or 5 soundtracks. For example, the user may have 
the option of selecting one of three variations on as many as five different 
instruments. The user may also have the option of selection 1 or 2 or more 
track-specific sound effects, and 1 or 2 or more global sound effects 
applicable to all tracks. 

[00026] Generally, the soundtracks and effects are made perceptible to a 
user of the handheld device, for example the soundtracks may be displayed 
or played audibly for selection by the user. In one embodiment, the 
soundtracks and effects are stored on the handheld device, and in another 
embodiment the soundtracks and effects are downloaded from some other 
location onto the device, either wirelessly or via a hardwired connection, by 
and upon selection thereof. 

[00027] In another embodiment, the first soundtrack is selected to play 
for a first time interval, and another soundtrack is selected to play for a 
second time interval different than the first time interval. The time intervals 
of the soundtracks may be overlapping or separate. 

[00028] In one embodiment, the soundtracks play the throughout the 
full interval of the audio mix, unless specified otherwise. In one 
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embodiment, the time interval is determined when the soundtrack is 
selected relative to previously selected soundtracks already playing. Thus 
while one or more soundtracks play, the user may add an additional 
soundtrack by selecting the new soundtrack and after a time interval 
remove the new soundtrack by de-selecting (merely re-selecting) it. 

[00029] Upon selecting the soundtracks and any effects and 
corresponding time intervals, the audio mix is saved. In one embodiment, 
the newly created audio mixes are saved in an audio format file, for 
example, the MIDI format audio file. The MIDI audio format has a time 
reference, with which start and stop times for corresponding soundtracks 
may be associated. 

[00030] FIG. 3 illustrates an exemplary soundtrack data set file 300 that 
is stored on the handset in one embodiment of the disclosure. The 
exemplary data set includes a plurality of sets of instruments, or some other 
melodic soundtracks, for example 310 and 320 in FIG. 3. In the exemplary 
embodiment each instrument or melodic soundtrack has three variations 
312, 314 and 316, but more generally there may be more or less. Only one 
variation, variation 322, of the second plurality of instrument sets is 
illustrated, but generally there are more- The instrument tracks could 
instead be some other distinct melodic soundtrack and/ or variations 
thereon. 
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[00031] The exemplary soundtrack data set file 300 also includes a 
plurality of sound effect soundtracks 330 including an effect soundtrack 332, 
334 and 336 for a corresponding one of the soundtracks 312, 314 and 316, 
respectively. Each of the sound effect tracks 332, 334 and 336 may be, for 
example, a reverberation soundtrack- Alternatively, there could be another 
set of effects, for example a corresponding echo track (not illustrated) for 
each of the soundtracks 312, 314 and 316. Generally, there are also, in some 
embodiments, corresponding effects soundtracks for each of the other 
soundtracks 322, etc. In some embodiments, there are no effect soundtracks. 
More generally, each soundtrack of the soundtrack data set file 300 in FIG. 3 
is a different soundtrack. 

[00032] The exemplary soundtrack data set file 300 also includes a 
corresponding reference or base soundtrack 302. In one embodiment, there 
is a plurality of unique soundtrack data set files each having a 
corresponding base track, e.g., Jazz, Blues, Rock, Pop, etc., and 
corresponding instrument variation soundtracks stored on the handset. 
Additional soundtrack data set files may be downloaded onto the device, 
for example, for a fee or as part of a subscription. 

[00033] In one embodiment, a polyphonic audio mix is created on the 
handset by entering reference data for corresponding soundtracks of a 
selected soundtrack data set file into an audio mix data reference file, for 
example, by selecting plural soundtracks at a user interface, as discussed 
above. The audio mix data reference file having soundtrack reference data 
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is representative of the user defined polyphonic audio mix. In one 
embodiment, the audio mix data reference file is not in an audio file format. 

[00034] The audio mix data reference file having the reference data is 
stored on the handset as a distinct file, separately from the soundtrack data 
set files. The audio mix data reference file is itself preferably devoid of 
sound track data. Thus many audio mixes may be mixed and saved in the 
form of audio mix data reference files by the user without substantial 
memory usage, which is often limited on mobile communication handsets, 
since only the audio mix data reference file having the reference data is 
created and stored. Tn other embodiments, the audio mix data reference file 
includes copies of the soundtrack data, but this requires comparatively 
substantia) memory for storage. 

[00035] In FIG. 4, the audio mix data reference file 400 includes a file 
name field 410, one or more global sound file effect fields 420, for example 
tempo and/ or key, and a plurality of soundtrack fields 430, 432, 434, etc* In 
embodiments where a base track is desired, the audio mix data reference file 
includes a base track field 450. 

[00036] The file name field 410 stores file name data designated by the 
user for the corresponding user defined polyphonic audio file. The file 
name data may be entered by the user, for example upon completion of 
mixing the soundtrack. 
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[00037] The global sound effect reference field 420 stores global effect 
reference data, which is communicated to the synthesizer when the 
polyphonic audio mix is played. More than one global effect reference may 
be stored in the field 420, or alternatively multiple reference data fields may 
be provided for storing corresponding sound effect references. In one 
embodiment the one or more sound effect fields provide the audio 
synthesizer with instructions for the sound effects that are to be played for 
the corresponding polyphonic audio mix. 

[00038] The base track field 450 stores base track reference data, which 
references the base track, if any, associated with the corresponding 
polyphonic audio mix. In one embodiment, the base track field provides the 
audio synthesizer with instructions for which base track is to be played for 
the corresponding polyphonic audio mix. 

[00039] The soundtrack reference fields 430, 432, etc. store soundtrack 
reference data, which references corresponding soundtracks associated with 
the corresponding polyphonic audio mix. In one embodiment the presence 
of particular soundtrack reference in the soundtrack fields indicates that the 
soundtrack has been selected, 

[00040] In some embodiments, time is associated with each of the 
soundtrack reference fields. The time may, for example, be measured as a 
delay from the time that the polyphonic mix is initiated or played. Thus the 
soundtrack referenced in each field is played at the corresponding time 

12 



PAGE 45/61 s RCVD AT 9/112005 5:42:03 PM [Eastern Daylight Time] * SVfcUSPTOtFXtf -6/26 * DNIS:2738300 ' CSID: 1 8475232350 * DURATION (mm-ss):14-1 8 



SEP- 01 -2005 16:45 FROM : MOTOROLA PCS IPD 18475232350 



TO:USPTO 



P. 46'61 



"Mobile Wireless Communication Handset 
With Sound Mixer and Methods Therefor" 
Atty. Docket No. CS90041 



AppL No. 09/971,381 
Substitute Specification 



associated therewith. A subsequent entry of a reference to the same 
soundtrack played previously may be used to toggle the soundtrack off. 
Thus the start and stop time for the soundtrack is entered as the soundtrack 
is selected and de-selected. 

[00041] In other embodiment, the audio mix data reference file 400 has 
a reference field corresponding to each of the soundtracks and effects in a 
corresponding soundtrack data set file. Each field of the audio mix data 
reference file 400 also has an on/ off code that is set or re-set upon selection 
of the corresponding soundtrack when the corresponding polyphonic mix is 
created. Thus, when it is desired to play the polyphonic mix, the audio mix 
data reference file 400 references only those soundtracks and effects that 
have been switched on for play by the audio synthesizer. 

[00042] According to another aspect of the disclosure, the audio mix 
data reference file 400 without soundtrack data is integrated with a 
corresponding soundtrack data set file 300 into a common audio format file 
for uploading from the handset, for example to another user. 

[00043] In one embodiment, the integrated audio mix data reference 
and the soundtrack data set files are integrated irreversibly. Thus the 
recipient of the integrated audio file may listen to the audio file as mixed 
originally, but cannot extract the soundtrack data set file 300 for making 
new audio files based on the soundtrack data set file. Providing the 
irreversibly integrated audio format file permits users to sample the 
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associated soundtrack data set file, limited to the corresponding mix, 
without providing full access thereto. 

[00044] While the present disclosure and what is considered presently 
to be the best modes thereof have been described in a manner that 
establishes possession thereof by the inventors and enables those of 
ordinary skill in the art to make and use the same, it will be understood and 
appreciated that there are many equivalents to the exemplary embodiments 
disclosed herein and that myriad modifications and variations may be made 
thereto without departing from the scope and spirit thereof- Thus the 
invention is to be limited not by the exemplary embodiments but by the 
appended claims. 

[00045] What is claimed is: 
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